Remaps introduction

A Remap is a data field whose values are directly dependent on the values of another field within the system. This section looks at the type of Remaps that can be created using Engine and discusses some of the uses of remapped data.

The Remap object makes it possible to decode data into more meaningful representations and easily create bandings of data to aid identification of trends and patterns.

Type of Remap

Remap fields allow new derived data to be created based on existing data within the system. They allow the following derived data to be created:

• Decodes
• Numeric Bandings (data grouped into equally spaced ranges)
• Quantile Bandings (data arranged into groups of equal numbers)
• Parametric Bandings (data arranged into groups based on the results of queries)

Decodes

Decodes allows one value to be decoded - or mapped - to another value. The new column has values in it that are themselves directly associated with the values in a source column. For example

• M become Male; F becomes Female.
• fiesta, Feista, Fiesta become Fiesta; primera, Primera, primira becomes Primera.

You should also provide a mapping for NULL values if you wish these to persist when specifying a Decode Remap. Failure to do this will mean that valid data with NULL values will be lost.

Numeric Bandings

Numeric bandings allow data to be grouped into equally sized or user-defined bands.

Equally sized:

• Sales are grouped by month into Sales for January, February, March...etc.

User-Defined:

• Age Band is created of Under 16, Under 21, 22 - 30, 31 - 40, 41 - 50, 51 - 64, Over 65s...

Quantile Bandings

A quantile remap allows the grouping of continuous numeric data into discrete classes, by equal division of population. The number of bands is specified, and then the data is automatically assigned in such a way that each band contains the same number of records.

For example:

• Income is banded into deciles (10 bands, all containing the same number of records). The first band contains the records that have an income in the top 10%. The last band contains the records having an income in the bottom 10%...

Parametric Bandings

Parametric remaps appear at first glance to be like any other column in that they appear to consist of discrete values and represent some underlying stored data.

However, they differ from other columns in that each discrete value represents not some physically stored data value, but the results of a query.

No physical data is stored for a parametric remap other than the query details that sit underneath each column value. A parametric remap is therefore defined by the set of queries that go to make it.

These remaps are not columns in the normal sense, since a record may fall into any, none or all of the filters. This can have subtle implications for their use in queries and crosstabs.

An example parametric remap might be a column called "Healthiness"

• The first band is called LOW and contains customers who like drinking and smoking, are not members of a gym and suffer from 4 or more colds a year.
• The second band is called MEDIUM and contains customers who are members of a gym, eat take-aways frequently and drink moderately.
• The third band is called HIGH and contains people who are non-smokers, members of a gym, participate in organized sports events and eat at least 5 portions of fruit and veg. a day.

Uses of Remaps

Remaps are used for the following:

• Applying mode readable names to coded values. For example CC: Credit Card, DD: Direct Debit, C: Cash...
• Saving space. By loading a value as an integer code and then decoding it to a legible string value, storage space can be reduced. For example 1: January, 2: February, 3: March. Rather than storing the data as a a text field, it is stored as a very small integer field.
• Adding new business dimensions to the system. For example
  • Postcodes into sales regions
  • Salesperson ID into JobID
• Grouping data into bands that can be analysed. For example,
  • Dates into Months, Quarters and Years
  • Age into Age Band
  • Income into Income Band
• Cleaning up data. For example
  • Tidying up job titles
  • Tidying up free text fields

Advantages of Remaps

The following lists some possible advantages of working with remaps

• Can reduce load time. It is quicker to create a remap on an integer code value than it is to import a large text field (assuming that the data does not have too many discrete values).
• Provides more efficient storage. When the number of discrete values is not too high it is more efficient to store the data as an integer, and then store a separate lookup table of the related string values than it is to store the string value for every row in the database. This is also often true of dates, where there can only ever be a maximum of 365 dates per year.
• Data Schema can be simplified as the need for lookup tables is removed. Although the lookup tables are still effectively there, they aren't visible to the user and so don't clutter up the data schema.
• New data can be derived rather than being imported. On the whole it is quicker and more efficient to create a derived column than it is to load an index a raw data field.

• Remaps
  • Remaps introduction
  • Remaps limits and constraints